The many and varied methods of human communication convey important information about individual and collective intent and desire, among myriad other emotions. The importance of automatic recognition of human emotion evidenced in speech and other forms of verbal and non-verbal communication has grown with the increasing role of speech/gesture human-computer interactions and application control. Human emotion provides a potent indicator of mood and attitude which, when considered in context, can improve customer experience in a retail setting or e-commerce application.
In addition to spoken language, humans communicate using many forms of non-verbal communication. In some forms, humans communicate by manipulating certain facial muscles, which in turn alter facial feature and form various expressions, such as smiles, frowns, and the like. Other forms of non-verbal communication include dynamic, time-varying head motions that are used to elicit many expressive head gestures, such as nodding. Isolating each such form of communication by capturing it using a sequence of images provides a richer understanding of a person's mental state. Not only do facial expressions offer robust avenues for automatic recognition of human emotion in adults, facial expressions could also prove useful for evaluating the mental states of young children who do not yet speak.
Often, the underlying feelings of people are subliminal, and not necessarily articulated, thus rendering the mood, thoughts, or mental state of a person difficult to ascertain. However, even when left unarticulated, subliminal mental states often affect how a person behaves and interacts with others on a given day. For example, an individual with a confused mental state may not be able to quickly and efficiently process information and may respond better to a slower or more repetitive explanations. Similarly, if a person had an impatient mental state, the person may respond better to a faster or terser explanation.
Additionally, analyzing the mental states of people can help to interpret individual or collective responses to surrounding stimuli. The stimuli can range from watching videos and sporting events to playing video games, interacting with websites, and observing advertisements, to name a few. Mental states run a broad gamut—from happiness to sadness, contentedness to worry, skepticism to certainty, and numerous others. These mental states are experienced either in response to the stimuli already mentioned or to everyday events: eagerness while awaiting an important telephone call, boredom while standing in line, or impatience while waiting for a cup of coffee. The Internet has only multiplied the ways in which an individual encounters stimulating information and circumstances.
Facial detection and interpretation of facial expressions under varying conditions is a task humans intuitively perform. In many cases, humans can ascertain mental state, identity, age, and gender simply by examining a person's face. The impression one receives from a person's displayed expression affects the tone and subject of interactions, conversation, and other actions. Humor and sympathy are just two examples of essential information conveyed through non-verbal facial expressions. Hence, the evaluation of mental states through facial analysis is performed by humans on a regular basis. In contrast, for computer based systems, identifying facial expressions is non-ordinary, and continues to evolve.
There are a growing number of applications which can benefit from the capability of automatically detecting human emotion. Applications for automated human emotion detection may include education, training, speech therapy, and analysis of media content to name a few.